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(54) An electronic camera having dual modes for previewing and capturing still images 



(57) An electronic camera uses a relatively more 
complex digital image processing technique in a still 
image mode to produce high quality still images, and a 
relatively more simple image processing technique in a 
motion preview mode to produce preview images of 
acceptable quality prior to initiation of the still image 
mode. The more complex digital technique is done in 
software in a general purpose processor section (35). 
while the more simple digital technique is implemented 
in a fixed digital circuit in an application specific inte- 
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grated circuit (27), which also implements timing and 
control functions. The motion preview mode uses a 
shorter image readout period than the still mode and fur- 
ther involves mapping image sensor pixels into a fewer 
number of color display pixels on a color LCD display 
(10). The mapping further converts color pixel signals 
from a mosaic array into a different color pattern on the 
color LCD display (10). 
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Description 

HELD OF THE INVENTION 

The invention pertains to an electronic still camera s 
for composing and capturing still images, and. mae par- 
ticularly, to an electronic camera having a "motion" mode 
for previewing a scene and a "stiir mode for capturing a 
particular image in the scene. 

10 

BACKGROUND OF THE INVENTION 

Consumer camcorders which include the capability 
of recording analog motion arxl/or still images on 8mm 
or VHS videotape have been developed by a number of 75 
companies. Motion images are recorded in the same 
manner as in any standard camcorder. These cameras 
include a single chip charge coupled device (CCD) sen- 
sor having a color fitter array that provides a spatially 
color-sanpled in^ge. To record still images, the user so 
pushes a "still capture" button at the desired instant. The 
image obtained from the CCD sensor is temporarily 
stored in a digital menrw)ry. The image is then read from 
the memory and recorded onto the videotape. Some 
camcorders include color liquid aystal displays (LCD), 25 
which are also spatially color-sampled devices. Some 
are relatively large, for exannple. ranging from approxi- 
mately 2.5** to 4" in diagonal. Such a display is used, 
instead of a normal eyepiece viewf inder, to allow the user 
to properly frame the subject and view the images as 30 
they are t>eing recorded. It is also used to view the 
recorded images as the videotape is played back. 

Rgure 1A shows a typical color LCD display, in 
which the liquid crystal material is trapped between an 
upper glass plate 1 and a lower glass plate 2. The upper 35 
plate 1 has a common transparent electrode 3 and an 
array 4 of color filters surrounded by a black masks. The 
lower plate 2 includes an array 6 of transparent pixel elec- 
trodes juxtaposed underneath the array 4 of color fitters. 
Individual pixel electrodes are activated via thin film tran- 4o 
sistors (TFT) 7 that are controlled from a video signal on 
the source lines 8 and a scanning signal on the gate line 
9. The LCD display includes the usual polarizer layers 
(not shown) on the glass plates 1 and 2, such that acti- 
vation of selected transparent pixel electrodes allows 45 
light to pass through the corresponding color filters and 
reflect to the viewer, thereby creating a color image. A 
typical LCD display such as the Epson LB 2F-BC00, 
manufactured by Seiko-Epson Company. Japan, has 
atx>ut 240 lines of pixels and about 300 pixels per line, so 
with an image aspect ratio of 4:3. Such an aspect ratio 
allowvs the entire area of the image obtained from the 4:3 
aspect ratio tslTSC format CCD sensor to be displayed 
on the LCD screen, so that the LCD screen composition 
will be the same as the image that is recorded by the 55 
camcorder NTSC format recorder, for later display on an 
NTSC format television display. Note that because the 
LCD has only 240 lines of pixels, the interlaced NTSC 
signal is displayed using a "repeat field" technique. 



where both the odd and even fields from the NTSC for- 
mat sensor are displayed using the same lines of pixels 
on the LCD. This LCD. like most commercially available 
LCDs, has "rectangular" pixels, rather than square pix- 
els, where the distance between pixels in the horizontal 
direction is for example 2/3 the distance in the vertical 
direction. The LCD pixels are overlaid with a diagonal 
RGB stripe pattern as shown in Figure 1 B. 

In camcorders, the processing for both the still 
images and the motion images is identical. Such 
processing is normally implemented by hardwired ana- 
log integrated circuits, although camcorders which use 
digital image processing integrated circuits have been 
produced. Such camcorders convert the signal from the 
CCD sensor into an NTSC composite or conrponent for- 
mat signal, which is provided to a video recording sub- 
system or a video output jack. The color LCD display 
includes circuitry todecode the NTSC conposite or com- 
ponent signal back into spatially subsampled RGB sig- 
nals to drive the individual RGB pixels on the LCD 
sensor. 

In a system oriented toward still photography, and in 
particular a digital still system, it would be desirable to 
avoid the necessity of generating an NTSC format signal 
in order to reduce the complexity of the required circuitry 
In a totally digital system, that is, both the recording and 
display channels are digital, it is further desirable to min- 
imize inconpatibiiity between thechannels. The problem 
is to achieve these objective in an architecture that min- 
imizes cost and complexity and maximizes user han- 
dling. 

SUMMARY OF THE INVENTION 

This problem is solved according to the invention by 
a number of features. In one aspect, the electronic cam- 
era is operable in a still image mode according to a rel- 
atively more complex digital image processing technique 
to produce high quality still images, and in a motion pre- 
view mode according to a relatively more simple digital 
image processing technique to produce a preview image 
of acceptable quality prior to initiation of the still image 
mode. Such an architecture is particularly adapted to 
mapping an array of color innage pixels from a sensor 
into an array of color display pixels on an LCD display 
comprising discrete LCD display pixels fewer in number 
tiian image sensor pixels. In tiiat case, a relatively simple 
digital processing technique combines same-colored 
image pixel signals into a fewer number of intermediate 
pixels that correspond to the arrangement of the color 
display pixels. 

The advantage of the invention is that the two modes 
can be tailored for a relatively low quality "motion" mode 
and a much higher quality "still" mode. The motion mode 
images from the CCD sensor are processed by a hard- 
wired digital signal processing circuit that generates low 
resolution, spatially subsanpled digital image data 
which can directiy drive the relatively low resolution LCD 
display. This reduces the conplexity and clock frequency 
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of the required circuitry, compared to generating an 
NTSC format signal, as is normally done in the prior art. 
The still mode image from the CCD sensor is processed 
by a general purpose processor (CPU) which executes 
an image processing software program in order to pro- 5 
duce a high quality digital still image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The invention will be described in relation to the to 
drawings, whereon 

Figures 1 A and 1 B show the structure and color fitter 
pattern of a known color liquid crystal display (LCD); 
Figure 2 is a block diagram of an electronic camera is 
incorporating dual modes for composing and cap- 
turing a still image according to the invention; 
Figures 3 A and 3B are diagrams of progressive scan 
image sensors useful with the camera of Figure 2; 
Figure 4 is a diagram of the Bayer color filter geom- 20 
etry for the sensor used with the camera of Figure 1 ; 
Figure 5 shows the line timing for the sbll mode of 
operation; 

Figures 6A and 6B show the line timing for the pre- 
view mode of operation; 2S 
Figure 7 shows the special line skipping pattern 
used in the preview mode; 
Figure 8 shows further detail of the preview mode 
processing circuit shown in Figure 2; 
Figures 9A and 9B show further detail of the image 30 
statistics processor shown in Figure 2; 
Figure 10 shows one example of still mode image 
processing; 

Figure 1 1 shows the effect of pixel mapping from a 
sensor to an LCD display, each having different 35 
aspect ratios; and 

Figure 1 2 shows an enhancement to the block dia- 
gram of Figure 2 in which a different sensor dock 
frequency is used in each of the dual modes. 

40 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

A block diagram of a camera incorporating dual 
modes of processing according to the invention is shown 45 
in Figure 2. The camera includes an electronic color dis- 
play for example, a color liquid crystal (LCD) display 10 
of the type shown in Figure 1 A, and a user control section 
12 having a number of user control buttons, including 
zoom buttons 1 4, a preview button 1 5 and a capture but- so 
ton 1 6. To take a still picture, the user turns on the camera 
(using a power switch (not shown), which may be auto- 
matically enabled when the user depresses the zoom 
buttons 14 or the preview button 15. or partially 
depresses the capture button 16). The user composes ss 
the picture by depressing the "zoom in" or "zoom out" 
buttons 1 4, and by adjusting the position of the camera, 
while observing the display image. When the user is sat- 
isfied with the compositon on the color LCD display 10. 



the user depresses the capture button 16. The camera 
then captures a single still image, firing a flash 18 if nec- 
essary when the anrtbient illumination level is low. The 
still image is focused upon an image sensor 20 by a 
motor driven zoom lens 22. The intensity of the image 
light upon the sensor 20 is regulated by a motor-driven, 
variable, mechanical aperture 24, while exposure time is 
regulated electronically by appropriate docking of the 
sensor 20. The still image from the image ser^br 20 is 
processed and digitally stored on a removable menK)ry 
card 26. 

Control of the sensor is provided by a timing and 
control section 27, which is an application specific inte- 
grated circuit (ASIC) with processing and timing func- 
tions, for both capture and preview operating nxxles. For 
instance, the timing and control section 27 includes a 
sensor timing circuit 28 for controlling the image sensor 
functions. The sensor timing circuit 28 provides the sig- 
nals to enable sensor drivers 30. which provide horizon- 
tal clocks (H1, H2). vertical clocks (VI . V2), as well as a 
signal FDG for activating a drain structure on the sensor 
20. The output of the image sensor 20 is amplified and 
processed in an analog gain and sanpling (correlated 
double sampling (CDS)) circuit 32. and converted to dig- 
ital form in A/D converter stage 34. The A/D output signal 
is provided to a processor section 35, which includes a 
digital processor 36 for temporarily storing the still 
images in a DRAM menrwry 38. The digital processor 36 
then performs image processing on the still images, and 
finally stores the processed images on the removable 
memory card 26 via a memory card interface circuit 40, 
which may use the PCMCIA 2.0 standard interface. An 
EPROM memory 42 is used to store the firmware which 
operates the processor 36. The components of the proc- 
essor 35 are interconnected through a data bus 43, 
which also connects to the timing and control section 27 
and to the card interface 40. 

The motor-driven zoom lens 22 includes a zoom 
motor 44, a focus motor 46, and an aperture motor 48 
(all controlled by lens motor drivers 50). The timing and 
control section 27 further includes a control interface 52 
connected to the lens motor drivers 50 and to a flash con- 
trol circuit 53 via a photosystem interface block 54, which 
controls the operation of the zoom lens 22 and the flash 
1 8. The lens zoom position is controlled by the photosys- 
tem interface block 54 based on position input from the 
zoom control buttons 14 through a user status and con- 
trol section 55. The focusing, exposure control, and white 
balance is done automatically, as is typically the case in 
consumer camcorders. This is done by computing 
"image statistics" in an image statistics processor 60 in 
the real-time ASIC (timing and control section 27) as pre- 
view images are continuously read out of the image sen- 
sor 20 . The computed values are then used by a program 
inrplemented in the digital processor 36. which decides 
how to adjust the focus motor, aperture, analog gain con- 
trol, and analog white balance controls via the control 
interface 52 and the photosystems interface 54 on the 
ASIC timing and control section 27. Although the digital 
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processor 36 and the control interface 52 are shown as 
being within two separate sections, in some implemen- 
tations the same conponent could be used to perform 
both of these functions (as well as other of the recited 
functions) . Sensor image data is passed to the processor 
section 35 through a high speed interface 56 in the timing 
and control section 27. The sensor image data is also 
directed to the color LCD display 10 through a preview 
mode processing circuit 58. 

The timing and control section (ASIC) 27 is operable 
in two modes, a relatively low quality "motion" mode and 
a much higher quality "stiir mode. In the motion mode, 
images from the sensor 20 are processed by the preview 
mode processing circuit 58; In the still mode, images 
from the sensor 20 are processed in the processor 35. 
The processor 35 is a software-driven digital processing 
system that is slower than the ASIC 27. The preview 
mode processing circuit 58 is a hardwired digital signal 
processing circuit (part of the ASIC 27) that generates 
low resolution, spatially subsampled digital image data 
which can directly drive the relatively low resolution color 
LCD display 10. This reduces the complexity and clock 
frequency of the required circuitry, compared to gener- 
ating an NTSC format signal, as is normally done in the 
prior art- The preview mode processing circuit includes 
a pixel remapper 62 for mapping the greater number of 
image pixels from the sensor 20 into the lesser number 
of display pixels (i.e.. corresponding to the array 6 of 
transparent pixel electrodes in Figure 1) in the color LCD 
display 10. The color saturation of the remapped pixels 
is then adjusted in a color adjustment circuit 63 and its 
output is applied to a multiplexer 64. The multiplexer 64 
selects image data either from the preview mode 
processing circuit 58. producing a preview image, or from 
the high speed interface 56. which allows for siitably pre- 
processed viewing of stored images. 

In this camera, the image processing used to create 
the preview mode is done in the timing and control ASIC 
27, since the processing must be done rapidly. About 60 
images per second are processed in preview mode. 
However, since the image quality of the displayed image 
is limited by the resolution and color gamut of the LCD 
screen of the LCD color display 10. there is no need for 
elaborate image processing. Therefore, sinple "preview 
mode" image processing is performed in a fixed digital 
circuit embedded in the preview mode processing circuit 
58 (which is part of the ASIC). The quality requirements 
for the still mode are much greater, since these images 
will be downloaded to a computer, and may be displayed 
on a high resolution CRT display, or printed on a high 
quality thermal printer. Therefore, the digital image 
processing must be more elaborate. By using the digital 
processor 36 to inplement software procedures stored 
in the firmware memory 42, complex procedures can be 
implemented. These procedures can take several sec- 
onds to complete, since real-time operation is not 
required. Use of firmware-stored software allows the still 
mode image processing to be upgraded without requir- 
ing a new ASIC design. In effect, what happens is that a 



relatively less complex digital image processing tech- 
nique is used in the motion preview mode, but at a higher 
data rate, and a relatively more complex digital image 
processing technique is used in the still mode, but at a 
5 slower data rate. 

Since the update rate, that is, the number of images 
that need to be supplied per unit time, is different for the 
still mode than for the motion mode, it is beneficial to use 
different dock frequencies for the different modes of 
10 operation. For example, as shown in Figure 1 2, a system 
oscillator 1 00 produces a 1 2 mHz clock frequency for use 
In the motion nrode to obtain more updates/second (e.g. . 
60 images per second), while a divider 102 divides by 
two to provide a 6 mHz clock frequency for the still mode. 
75 The tower frequency allows more time to accurately posi- 
tion the damp and sample pulses so as to avoid CCD 
output signal transitions. This increases noise immunity 
in the still mode. A multiplexer 1 04 is enabled by the con- 
trol interface 52 to determine which dock frequency is 
20 applied to the sensor timing drcuit 28. Though not spe- 
cifically shown, the changed timing is also communi- 
cated to the A/D stage 34 and other tming and control 
drcutts. 

The sensor 20 is a progressive scan interline image 
25 sensor having a noninterlaced architecture, as shown in 
more detail in Figure 3A. The sensor comprises a two- 
dimensional array of photosites 66, e.g. pholodiodes. 
arranged in rows and columns of image pixels, a plurality 
of vertical registers 68 adjacent photosite columns for 
30 transfen-ing rows of image pixel charge from the pho- 
tosites 66 to a horizontal register 70 for readout respon- 
sive to dock signals from the sensor drivers 30, and a 
charge drain (specifically, a fast dump structure 72) inter- 
posed between the output of the vertical registers 68 and 
35 the horizontal register 70 for eliminating complete rows 
of image pixels at a time from the image sensor 20. A 
preferred image sensor is the Kodak mode! CCD KAI- 
04O0CM image sensor, which has approximately 512 
active lines vwth approximately 768 active pixels per line 
40 and an image aspect ratio of 3:2. This sensor is 
described in a Performance Specification document 
available from Eastman Kodak Company, Rochester. 
New York. Each pixel is 9 microns "square", since both 
the vertical and horizontal distances between the centers 
45 of adjacent pixels are 9 microns. The 3:2 image aspect 
ratio of the CCD sensor, although wider than the 4:3 
aspect ratio of the display, is considered to be a preferred 
aspect ratio for still photography, in that the standard 
35mm film format, and standard 4R (4" x 6^ prints also 
50 have a 3:2 image aspect ratio. The sensor uses a color 
f ilter array pattern known as the "Bayer checkert^oard" 
pattern, described in U.S. patent 3,971.065. which is 
shown in Figure 4. Such a color filter an-ay is character- 
ized by a mosaic pattern in which the filter colors alter- 
55 nate in both line and column directions. In the normal 
operating mode, all of the image pixels on the sensor are 
transfen-ed as color image pixels to the horizontal regis- 
ter 70, which delivers a stream of color pixel signals to 
the analog gain and CDS circuit 32 (see Figure 2). The 
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color pixel signals are subsequently converted to digital 
pixel signals in the A/D converter 34. 

The sensor 20 uses a progressive scan readout 
method, which allows the entire image to be read out in 
a single scan. The accumulated or integrated charge for 
the photodiodes comprising the pixels 66 is transported 
from the photosites to light protected vertical (parallel) 
registers 68 by applying a large positive voltage to the 
phase-one vertical clock (V1). This reads out every row, 
or line, into the vertical registers 68. The image pixel 
charge is then transported from the vertical registers 68 
to the horizontal register 70 by two-phase clocking of the 
vertical clocks (VI, V2). Between the vertical and hori- 
zontal registers is the fast dump structure 72. which is 
further described in the Performance Specification doc- 
ument for the KAI-O400CM sensor. By setting a suitable 
positive potential on a fast dump gate line FDG, charge 
from the row of pixel values currently adjacent to the fast 
dump structure 72 is transferred from the CCD directly 
into the sensor siA>strate 74 rather than to the horizontal 
register 70. This dunp, or line clear, is accomplished dur- 
ing the vertical-to-horizontal transfer time. When prop- 
erly controlled by the sensor timing circuit 28. the fast 
dump structure 72 allows lines of charge to be elimi- 
nated. 

The timing and control section 27 operates the elec- 
tronic camera shown in Figure 2 in the two aforemen- 
tioned nrKxJes, including a first "still" mode wherein all 
rows of image pixel charge con'esponding to each line 
are progressively read out through the horizontal register 
70 during a single scan, and a second "motion" mode 
wherein some of the rows of image pixel charge corre- 
sponding to some lines are eliminated through the fast 
dump structure 72 prior to readout. As applied to the 
embodiment of Figure 2, the first mode conresponds to 
a high quality still imaging mode while the second mode 
conresponds to a special line skipping" mode for driving 
the color LCD display 10. In the second mode, the timing 
and control section 27 controls the fast dump structure 
72 to eliminate two or more consecutive lines of image 
charge from the image sensor 20 for every one or more 
lines of image charge that are transferred to the horizon- 
tal register 70 for readout, thereby generating a pattern 
of lines (shown in Figure 7) suitable for driving the LCD 
display in a "repeat field" mode. An appropriate video sig- 
nal which displays the entire 32 aspect ratio sensor 
image on the 4:3 aspect ratio LCD. without introducing 
geometric distortion, is generated by alternately elimi- 
nating two or four consecutive lines of Image charge for 
every pair of lines of inr^ge charge that are transferred 
to the horizontal register 70. 

The sensor timing circuit 28 is controlled by the con- 
trol interface 52 to provide the clock signals V1 . V2, HI , 
H2, and the gate signal FDG according to the two modes 
of operation. The timing signals for the first mode are 
shown in Figure 5; those for the second mode are shown 
in Figure 6a and 6b. The two-phase cycling of signals VI 
and V2 control the transfer of lines of image pixel charge 
from the vertical registers 68 to the horizontal register 



70. The two-phase cyding of signals HI and H2 control 
the transfer of a stream of color pixel signals from the 
horizontal register 70 to sut>sequent circuits in the cam- 
era. The level of the signal FDG determines whether the 
5 image charge is dumped to the substrate 74 or trans- 
fen-ed to the horizontal register 70. When the sensor 20 
is clod^ using the first timing nrxxle shown in Figure 5 
for all lines of the sensor, all lines of the sensor are 
clocked out, one after the other, through the horizontal 
10 register 70, processed in subsequent camera circuitry, 
and stored in the renrravable mentory 26. This timing 
mode provides a high quality progressive scan still 
image, but may take 1/30 second or longer to read out 
the still image. Such timing, however, is acceptable for 
75 still mode usage, and. as mentioned before, does not 
require unusually high speed components and, indeed, 
may benefit from a kjwer speed clock. 

To provide an image to the color LCD display 10. a 
lower resolution image is suitable, but the upcteite rate 
20 must be sufficient to provide good motion rendition and 
eliminate display flicker. Moreover, the sensor 20 
includes the aforementioned array of color filters 
arranged in a particular mosaic color pattern (e.g.. the 
checkerboard Bayer pattern of Fig. 3). and the lines of 
25 image charge that are transferred to the horizontal reg- 
ister 70 should preserve that particular color pattern in 
the pattern of lines that are generated for the line-skip- 
ping readout. To provide this kind of image, the sensor 
is read out in the second mode as shown in Figure 7, 
30 using the timing shown in Figures 6A and 6B. As shown 
in Figure 6A. the first two lines (1 and 2) are read out as 
in the normal mode. These provide a green-red and a 
blue-green line. The next two lines (3 and 4) are elimi- 
nated by turning on the fast dump structure 72 during the 
35 time that these lines are transferred past the fast dump 
structure 72. Next, as shown in Figure 6B. lines 5 and 6 
are read out normally, and then lines 7, 8, 9. and 10 are 
eliminated. Next, the Figure 6A timing is used to read out 
lines 1 1 and 12. while eliminating lines 13 and 14, and 
40 then the Figure 68 timing is used to read out lines 1 5 and 
16. while eliminating lines 1 7-20. This process proceeds 
for the entire image readout period, during which 102 
pairs of lines are read out. and 1 54 pairs of lines are elim- 
inated. 

45 This special "line skipping" readout mode, as shown 
in Figure 7. allows the sensor 32 aspect ratio image to 
be fully displayed on a 4:3 aspect ratio LCD without "geo- 
metric distortion", that is, without stretching the image 
vertically and without cropping off the horizontal edges 

50 of the image from the image sensor. This allows the LCD 
to properiy show the entire 3:2 aspect ratio image cap- 
tured by the sensor, so that an image can be properiy 
composed. 

As the 512 lines of the CCD imager are read out 
55 using the special line skipping" mode, they are dis- 
played using only 204 out of the approximately 240 LCD 
lines of pixels. The remaining approximately 36 lines can 
either be masked behind a bezel, so that they are not 
visible, or preferably may be used to display status infor- 
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mation. such as the time-of-day, image number, or a 
"push-button menu" for the user buttons. Figure 11 
shows a useful application Of such conversions. A sen- 
sor having a 3:2 aspect ratio is shown mapped into an 
image area 90 of an LCD display screen 92 having a 4:3 ; 
aspect ratio. A proportional renoapping leaves a status 
area 94 available for other purposes, specifically to show 
text indicating the function of a set of reconf igurable con- 
trol buttons 96 in the control section 12. The function of 
the buttons is specified by the user status and control 
section 55 (Figure 2). This status information graphics 
data can be supplied by the digital processor to the LCD 
1 0 via high speed interface 56. when the MUX 64 is con- 
trolled so as to use the digital data from interface 56, 
rather than from circuit 58, for supplying data to the final 
approximately 32 lines of the display 10. 

The ''line skipping" readout causes some minor ver- 
tical sampling artifacts, but these are not noticeable in 
the small LCD displays. The pixels output for the sensor 
20 in line Skipping mode continue to have the Bayer-type 
color filter repeating pattern, so that they can be proc- 
essed using processing techniques designed for the 
Bayer pattern. 

The processing conrplexity of the camera of Figure 
2 is considerably sinrplif ied by directly mapping the RGB 
sensor pixels 66 to the RGB pixels of the display 10. The 
easiest way to do this for the image sensor 20 (512 lines 
X 768 pixels and 3:2 aspect ratio) is to have an LCD dis- 
play with 51 2 X 768 pixels and the same aspect ratio and 
color filter pattern. However, this would be a custom, 
costly LCD. Instead. LCDs have fewer display pixels than 
image pixels on the image sensor, normally have a 4:3 
image aspect ratio, and use the diagonally striped RGB 
pattern shown in Figure 1 B. In this discussion, an LCD 
pixel an-ay of about 240 lines x 312 pixels per line is 
assumed. 

Therefore, the sensor pixels are processed in a 
"pixel mapping circuit", such as the LCD pixel remapper 
62. A block diagram of this circuit is shown in Figure 8. 
Note that there are 768/2 = 384 green or red/blue pixels 
per line on the sensor (see Figure 4). There are about 
300/3 =100 green, red, or blue pixels per line on the LCD 
(see Figure IB). Thus, there are approxinrately 1/4 as 
many LCD pixels per line (per color) as there are sensor 
pixels per line. Therefore, the basic plan is to combine 
same-colored image pixels into a fewer nunr^er of inter- 
mediate, combination pixel signals that are then mapped 
into the color display pixels. For instance, a simple "pixel 
mapping" circuit maps four green sensor pixel signals 
into one green LCD pixel for one line by summing two 
green sensor values, spaced apart by 4 CCD pixel posi- 
tions, in the green pixel summer 76 and dividing by two 
via bit-shift wiring. The necessary delay is provided by 
the registers 82 clocked at one fourth the pixel rate, fur- 
ther delayed by one pixel clock It also maps four red sen- 
sor pixels into one LCD pixel in the same manner (using 
the red/blue summer 78). and also stores this value in a 
100 pixel FIFO 80. The FIFO 80 compensates for the fact 
that the sensor has line sequential red and blue pixels, 



by supplying blue pixels on the red sensor lines, and red 
pixels on the blue sensor lines. Four pixel delays are pro- 
vided by the registers 82' clocked at one fourth pixel rate 
(CLK/4). The mapping process is basically, therefore, a 
5 process which, in its simplest form, involves averaging 
of signals to produce a smaller number of output color 
pixels than input color pixels. (The CFA interpolation 
algorithm discussed In reference to Rgure 10, on the 
other hand, produces a larger number of "output" color 
70 pixels than input color pixels.) Alternate groups of 2 or 4 
lines of sensor values are discarded during preview 
mode by using the fast dump gate shown in Figure 3A. 
as described in connection with the line skipping" mode. 
This allows the sensor readout time to be decreased by 
75 more than 1/2 during the preview mode. 

Another feature of the design is that by removing the 
NTSC rate driving circuitry from the color LCD display 
10. the active matrix LCD can be updated at a slower 
frequency than is normally used. This reduces the cost 
20 and power consuntption off the LCD driver circuits (not 
shown). For example, the LCD can be updated at 30 Hz 
(provided the LCD active matrix display is designed so 
as not to exhibit noticeable flicker at this update rate), 
instead of 60 Hz. 
25 Once the LCD pixel values are calculated, the LCD 
color adjust circuit 63 increases the color saturation of 
the image by forming R-G and B-G color difference sig- 
nals, and adding or subtracting afraction of these signals 
in an array 84 of adders and subtracters from the original 
30 RGB signals in order to increase the color saturation of 
the displayed image. This circuit performs a similar func- 
tion as a 3x3 color matrix, but uses less hardware and 
provides less color accuracy. The color accuracy is not 
aitical for the LCD display, however, due to the limited 
35 color reproduction quality of such displays. The color 
reproduction of the still image is much more important, 
and is done with a more complicated and precise color 
correction method with the stored firnrrware in EPROM 
42. 

40 Rgures 9A and 9B show the processing in the image 
statistics processor 60, which computes real-time values 
for both still and motion image capture, all during the pre- 
view mode. These values include 24 R, G. and B aver- 
ages used to set white balance and exposure, and 6 

45 average high frequency "detail" values used to set focus. 
The 24 RGB averages for white balance are calculated 
for a group of 4 x 6 rectangular image regions, for each 
of R. G, and B; a block diagram of the calculation in one 
color is shown in Figure 9A. The 6 average "detail" values 

50 for focusing are calculated for green pixels only by accu- 
mulating the absolute value of the differences between 
nearby green values: a block diagram of this calculation 
is shown in Rgure 9B. These values are computed for 
each preview image and downloaded to the processor 

55 36. The processor 36 implements a firmware stored pro- 
cedure which determines the optimum exposure param- 
eters (exposure time, f/stop, and analog gain), white 
balance settings, and lens focus setting. For the still 
mode, the processor 36 also decides, based on the last 
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preview images, whether to fire the flash, and deter- 
mines the optimum exposure parameters, white balance 
settings, and lens focus setting for the still mode. 

Once the still image is captured, the digital proces- 
sor 36 implements the stored firmware procedures to s 
process and store the still image. Figure 1 0 shows a dia- 
gram of one possil)le still Image processing method. The 
CFA interpolation diagram may include the green inter- 
polation method desaibed in U.S. patent application 
USSN 085.520. filed June 30. 1993. in the name o1 the io 
same assignee as the present application, and the 
chrominance interpolation method described in U.S. pat- 
ent 4.642,678. both of which are incorporated herein by 
reference. The color matrix, tone correction, and edge 
enhancement st^ may be similar to those described 
in U.S. patent 5. 1 89.51 1 . also Incorporated herein by ref- 
erence. The image compression method may be the 
JPEG standard compression technique. 

The foregoing description envisions taking a single 
still picture following the motion preview mode. The cam- 
era can also optionally capture "bursts" of Ngh quality 
still images into the DRAM meniory during the "stilP 
mode, which are then processed as shown in Figure 10. 
Owing to the flash recharge time and other limitations of 
the Txjrst" mode, the "burst" nuxJe could utilize different 
exposure parameters (exposure time, aperture, analog 
gain, flash, and digital processing) than either the motion 
or the single still mode. 

The invention has been described with reference to 
a preferred embodiment. However, it will be appreciated 
that variations and modifications can be effected by a 
person of ordinary skill in the art without departing from 
the scope of the invention. For instance. Figure 3B shows 
a progressive scan sensor with two readout registers 86 
and 88 (which corresponds to the Performance Specifi- 
cation document for the Kodak KAI-031CM image sen- 
sor; the preferred embodiment of Rgure 3A simply uses 
but one register). The purpose is to eliminate the FIFO 
line delay 80 in the LCD pixel remapper 62. The pairs of 
lines read out by the registers include both a green/red 
line and a blue/green line. Therefore, by adding an ana- 
log multiplexer between the outputs of the two channels 
and the analog gain and CDS block 32, which is switched 
at 1/2 the sensor horizontal clock rate, it is possible to 
obtain a GRBG sequence of sensor data values at the 
output of the A/D stage 34. The LCD pixel remapper 62 
can then be designed to map from the CCD sensor color 
pixel pattern to the required RGBRGB LCD pixel pattern, 
without using a line delay. Since two CCD lines are read 
out in parallel, for each LCD line, fewer lines are elimi- 
nated via the fast dump gate than for the single-register 
sensor shown in Figure 3A. 

In particular, if the dual register CCD in Figure 3B 
had 51 2 x 768 square pixels with a 3:2 aspect ratio, and 
the LCD had approximately 240 display lines and a 4:3 
aspect ratio, the CCD readout would Involve fast dump- 
ing one pair of lines for every four pairs of lines read out 
from the CCD sensor. The pixel readout procedure for 
the horizontal registers can then be varied depending on 



the nrKxJe of operation: both registers are used for the 
motion imaging mode and one register Is used for the 
still imaging mode. Furthermore, although the Bayer pat- 
tern was described, other mosaic-type filter patterns 
could be used to advantage, for example, complemen- 
tary patterns involving cyan, magenta, yellow and green 
filters. The processing for the LCD pixel remapper 62 and 
the LCD color adjust circuit 63 would be accordingly 
modified to account for the different color anangement. 
The invention is summarized as follows: 



1. An electronic camera operable In a still image 
mode and in a motion preview mode, the improve- 
ment wherein 

IS the still image mode uses a relatively more 

complex digital Image processing technique to pro- 
duce high quality still Images, and the motion pre- 
view mode uses a relatively more simple digital 
Image processing technique to produce a preview 
20 image of acceptable quality prior to initiation of the 
still image nrxxle. 

2. An electronic camera as In 1 wherein both modes 
operate on color Images. 

3. An electronic camera as in 2 further Including an 
25 image sensor having an array of color image pixels 

and a display having a smaller anay of color display 
pixels, and wherein the Image processing technique 
used during the motion preview nfiode nraps the 
array of color image pixels into the smaller array of 
30 color display pixels. 

4. An electronic camera as In 3 wherein the image 
processing technique used during the motion pre- 
view mo6e also modifies the saturation of the color 
Image pixels. 

35 5. An electronic camera as in 3 wherein the display 
Is a liquid crystal display (LCD) comprising discrete 
LCD display pixels, and the Image processing 
method used during the motion preview nrrade maps 
nrwe than one image pixel from the sensor to gen- 
40 erate each value for Individual LCD display pixels. 

6. An electronic camera as In 1 wherein the more 
complex image processing technique implemented 
in the still Image mode is implemerrted In software, 
and more sinnple image processing technique imple- 

45 merited in the motion preview mode Is implemented 
in an application specific integrated circuit. 

7. An electronic camera operable in a still image 
mode and in a motion preview mode, said electronic 
camera comprising: 

so an image sensor including a two-dimensional 

array of image pixels covered by a mosaic pattern of 
color filters, said image sensor providing a stream 
of color pixel signals at an output thereof suitable for 
still inrmging; 

55 an analog-to-digital stage for converting the 

color pixel signals to digital pixel signals: 

a color display comprising a discrete two- 
dimensional arrangement of color display pixels, 
said color display having fewer color display pixels 
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than the number of image pixels on the image sen- 
sor; and 

a preview mode processor lor mapping the 
digital image pixel signals into at least a viewable 
portion of the color display pixels by combining 5 
same-colored image pixels into a fewer number of 
intermediate pixels that con-espond to the an^ange- 
ment of the color display pixels. 

8. An electronic camera as in 7 wherein the mosaic 
pattern of color filters covering the image sensor is 10 
different than the arrangement of color display pix- 
els. 

9. An electronic camera as in 8 wherein said mosaic 
pattern of color filters is the following pattern 

R G R G 15 

GBQB 

RGRG 

GBGB 

and wherein the preview mode processor combines 
the digital pixel signals into a fewer number of RGB 20 
pixels by averaging two or more of the sanDe-colored 
image pixels in each line to provide averaged display 
pixels. 

1 0. An electronic camera as in 9 wherein the preview 
mode processor further nxxitfies the saturation of 2S 
the averaged display pixels by adding or subtracting 
portions of one or more of the other colors to each 
averaged display pixel. 

11 . An electronic camera as in 7 wherein the image 
sensor and the viewable portion of the color display 30 
have the same aspect ratios such that substantially 

all of the image pixel signals can be mapped into 
substantially all of the color display pixels. 

12. An electronic camera as in 7 wherein the image 
sensor and the color display have different aspect 35 
ratios such that substantially all of the image pixel 
signals are mapped into the viewable portion of the 
color display pixels, leaving another portion of the 
color display pixels available for non-imaging use. 

13. An electronic camera operable in a still image 40 
mode and In a motion preview mode, said electronic 
camera comprising: 

an image sensor comprising a two-dimen- 
sional color filter array, a two-dimensional array of 
image pixels arranged in rows and columns with 45 
respect to tiie color filter array, and a horizontal reg- 
ister for outputting rows of color pixel signals; 

an analog-to-digitai stage for converting the 
color pixel signals to digital pixel signals; 

a color display comprising a specific two- so 
dimensional color pattern of display pixels arranged 
in rows and columns, having substantially fewer 
rows and fewer columns than the image sensor; 

a still image processor for processing tiie dig- 
ital pixel signals obtained during the still image 55 
nxxie, and 

a preview mode processor for mapping the 
digital pixel signals obtained during the motion pre- 
view mode into tiie display pixels according to a pro- 



cedure that digitally maps tiie digital pixel signals 
corresponding to the color filter array of tiie sensor 
into the color pattern of the color display. 

14. An electronic camera as in 13 wherein the pre- 
view mode processor further modifies tiie saturation 
of the digital pixel signals. 

15. An electronic camera as in 13 wherein the still 
image processor utilizes a software procedure for 
processing tiie digital pixel signals and the preview 
mode processor includes a fixed digital circuit 
embedded in an application specific Integrated cir- 
cuit (ASIC) for processing tiie digital pixel signals. 

16. An electronic camera as in 1 5 wherein the ASIC 
processes the digital pixel signals more rapidly than 
the software procedure. 

17. An electronic camera as in 13 wherein the color 
filter array covering the image sensor is different 
than the color pattern of display pixels. 

18. An electronic camera as in 17 wherein said color 
filter anray has the following pattern 

RGRG 
GBGB 
RGRG 
GBGB 

and wherein tiie preview mode processor combines 
the digital pixel signals into a fewer number of RGB 
pixel signals by averaging two or more of the same- 
colored image pixel signals in each line to provide 
averaged RGB pixel signals. 

19. An electronic camera operable in a still image 
mode and in a motion preview mode, said electronic 
camera comprising: 

an image sensor conprising a two-dimen- 
sional array of image pixels arranged in rows and 
columns, and a horizontal register responsive to 
clock signals for outputting rows of image pixel sig- 
nals; 

an analog-to-digrtat converter for converting 
tiie output image pixel signals into digital image pixel 
signals; 

a still image processor for processing the dig- 
ital image pixel signals obtained during tiie still 
image mode; 

a motion preview processor for processing 
tiie digital image pixel signals obtained during tiie 
motion preview mode; and 

a timing generator for operating the image 
sensor according to respectively different clock sig- 
nal frequencies depending upon which operating 
mode is being used. 

20. An electi-onic camera as in 19 wherein the clock 
frequency for operating tiie image sensor in the 
motion preview mode is higher than the clock fre- 
quency for operating the image sensor in the still 
image mode. 

21 . An electronic camera operable in a still image 
mode and in a motion image mode, said electronic 
camera comprising: 

an image sensor including a two-dimensional 
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array of image pixels and a horizontal readout sec- 
tion for providing an output stream of image pixel sig- 
nals; 

an analog-to-digital stage for converting the 
output image pixel signals into digital image pixel 5 
signals: and 

a timing and control section for driving the 
image sensor to provide the output stream of image 
pixel signals according to a pixel readout procedure 
involving clock signals, said pixel readout procedure 10 
being different for the two modes. 

22. An electronic camera as in 21 wherein the pixel 
readout procedure comprises separate clock signal 
rates for the still image mode and the motion image 
mode. '5 

23. An electronic camera as in 22 wherein the dock 
signal rate for the still image mode is slower than for 
the motion image mode. 

24. An electronic camera as in 21 wherein the hori- 
zontal readout section comprises two horizontal rea- 20 
dout registers and the pixel readout procedure 
comprises use of one horizontal register for the still 
image mode and the use of both horizontal registers 

for the motion image mode. 

25. An electronic camera as in 1 wherein the still 25 
image mode includes the capture of a single image 

or the capture of a burst of images. 

Claims 

30 

1 . An electronic camera operable in a still image mode 
and in a motion preview mode, characterized in that 

the still image mode uses a relatively more 
complex digital image processing technique to pro- 
duce high quality still images, and the motion pre- 3S 
view mode uses a relatively more simple digital 
image processing technique to produce a preview 
image of acceptable quality prior to initiation of the 
still image mode. 

40 

2. An electronic camera as claimed in claim 1 wherein 
both noodes operate on color images. 

3. An electronic camera as claimed in claim 2 further 
including an image sensor (20) having an array of 45 
color image pixels and a display (10) having a 
smaller array of color display pixels, and wherein the 
image processing technique used during the motion 
preview mode maps the array of color image pixels 
into the smaller array of color display pixels. so 

4. An electronic camera as claimed in claim 3 wherein 
the image processing technique used during the 
motion preview mode also modifies the saturation of 
the color image pixels. 55 

5. An electronic camera as daimed in claim 3 wherein 
the display (1 0) is a liquid crystal display (LCD) com- 
prising discrete LCD display pixels (6). and the 



image processing method used during the motion 
preview mode maps more than one image pixel (66) 
from the ser^or (20) to generate each value for indi- 
vidual LCD display pixels. 

6. An electronic camera as claimed in daim 1 wherein 
the more complex image processing technique 
implemented in the still image mode is implemented 
in software, and more simple image processing 
technique implemented in the motion preview mode 
is implemented in an application specific integrated 
drcuit 

7. An electronic camera operable In a still Image mode 
and in a motion preview mode, said electronic cam- 
era comprising: 

an image sensor (20) comprising a two- 
dimensional color filter anray. a two-dimensional 
array of image pixels (66) anranged in rows and col- 
umns with respect to the color filter array, and a hor- 
izontal register (70) for outputting rows of color pixel 
signals; 

an analog-to-digital stage (34) for converting 
the color pixel signals to digital pixel signals; 

a color display (10) comprising a specific two- 
dimensional color pattern of display pixels (6) 
arranged in rows and columns, having substantially 
fewer rows and fewer columns than the image sen- 
sor: 

a still image processor (35) for processing the 
digital pixel signals obtained during the still image 
mode, and 

a preview mode processor (58) for mapping 
the digital pixel signals obtained during the motion 
preview mode Into the display pixels according to a 
procedure that digitally maps the digital pixel signals 
corresponding to the color filter array of the sensor 
(20) into the color pattern of the color display (10). 

8. An electronic camera as claimed in daim 7 wherein 
the preview mode processor (58) further modifies 
the saturation of the digital pixel signals. 

9. An electronic camera as claimed in daim 7 wherein 
ttie color filter array covering the image sensor is dif- 
ferent than the color pattern of display pixels. 

10. An electronic camera as claimed In daim 9 wherein 
said color filter an-ay has the following pattern 

RGRG 
GBGB 
RGRG 
GBGB 

and wherein the preview mode processor (58) com- 
bines the digital pixel signals into a fewer number of 
RGB pixel signals by averaging two or more of the 
same-colored image pixel signals in each line to pro- 
vide averaged RGB pixel signals. 
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(54) An electronic camera having dual modes for previewing and capturing still images 



(57) An electronic camera uses a relatively more 
complex digital image processing technique in a still 
image mode to produce high quality still images, and a 
relatively more simple image processing technique in a 
motion preview mode to produce preview images of 
acceptable quality prior to initiation of the still image 
mode. The more complex digital technique is done in 
software in a general purpose processor section (35). 
while the more simple digital technique is implemented 



in a fixed digital circuit in an application specific irrte- 
grated circuit (27), which also inrtplements timing and 
control functions. The motion preview mode uses a 
shorter image readout period than the still mode and 
further involves mapping image sensor pixels into a 
fewer nunrtoer of color display pixels on a color LCD dis- 
play (10). The nnapping further converts color pixel sig- 
nals from a mosaic an^ay into a different color pattern on 
the color LCD display (10). 
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